#Tokenizer-API
An API to interface a tokenizer with tantivy.
The API will be kept stable in order to not break support for existing tokenizers.
#Tokenizer-API
An API to interface a tokenizer with tantivy.
The API will be kept stable in order to not break support for existing tokenizers.